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(57) Abstract: A method of returning to a program that is being received on a first channel is disclosed. The method includes storing 
data that characterizes at least one portion of the program. After storing this data, the user is free to change channels. The system 
automatically switches back to the first channel when data that characterizes a subsequent portion of the program matches the stored 
data. 
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free to change channels as desired. After a predetermined time (e.g., 60, 90, or 120 seconds), 
the timer times out, and the system automatically switches back to the stored channel. While 
the EP 366001 A2 system prevents users from being carried away watching an alternate 
program, it cannot correct the problem of returning a viewer to the original channel to soon 
5 or too late, since the decision to return to the original channel is based on elapsed time. A 
solution that would return the viewer to the original program only when the commercial has 
actually ended would be highly desirable. 



10 It is an object of the inventor to provide an improved channel selection system. 

To this end, the invention provides an automatic channel selection as defined in the 

independent claims. The dependent claims define advantageous embodiments. 

The present invention enables a user to store data that characterizes a portion 

of a program, and to subsequently change channels as desired. When data that characterizes a 
15 subsequent portion of the program matches the stored data, the channel switches back to the 

original program. 

More specifically, in one embodiment, while the user is watching or listening 
to a first program that is being broadcast on a first channel, the user indicates interest in the 
first program either explicitly (e.g., by pressing a button) or tacitly (e.g., by measuring dwell 

20 time). After the indication of interest, the user is free to change channels as desired, and to 
watch or listen to other programs. Meanwhile, the system monitors the original channel. 
When content that matches the first program reappears on the first channel, the system 
automatically reverts to the first channel. 

One aspect of the present invention relates to a method of returning to a 

25 television program being received on a first channel. The method includes the steps of 

accepting a storage command from a user, and storing data that characterizes the program in 
response to the storage command. The user is then permitted to change channels. After the 
user has changed channels, the method detects when data that characterizes a second portion 
of a broadcast on the first channel matches the stored characterizing data. When a match is 

30 detected, the television is returned to the first channel. 

Another aspect of the present invention, which is not limited to television, 
relates to a method of returning to a first program being received by a receiver via a first 
channel. The method includes the steps of accepting a storage command from a user, and 
storing data that characterizes the program in response to the storage command. The user is 



WO 01/62002 PCT/EP01/00514 

4 

memory and random access memory (not shown). The implementation of such 
microprocessor-based controllers is well known to persons skilled in the relevant arts. 
Alternatively, the controller 20 may use hard-wired logic to implement the process steps 
described below in connection with Figs. 3 and 4. 
5 Preferably, the controller 20 receives command codes 25a from a remote 

control unit 25. Fig. 2 illustrates a suitable keyboard arrangement for the remote control 25. 
The command codes 25a may be transmitted from the remote control 25 to the controller 20 
using any suitable approach including, for example, transmitting a unique sequence of 
infrared pulses from the remote control 25 to the controller 20 in response to the pressing of 

10 each key 26, 27a-27d, 28a-28b on the remote control 25. A wide variety of alternative 
arrangements for implementing control of the controller 20 can be readily envisioned, 
including, for example, using radio frequency or ultrasonic transmissions, or a wired 
connection between the remote control 25 and the controller 20. 

The hardware configuration of the remote control 25 itself is preferably similar 

15 to the hardware configuration of conventional remote control units. Any suitable arrangement 
of conventional remote control keys may be included on the remote control 25. For example, 
the illustrated remote control 25 includes control buttons for power 27a, mute 27b, volume up 
and down 27c-d, and channel up and down 28a-b. 

Returning now to Fig. 1 , when the controller 20 receives a command code 25a 

20 from the remote control 25 that corresponds to any of the conventional remote control 

buttons 27a-d, 28a-b the controller 20 sends appropriate commands to the main tuner 30 or to 
the monitor 31 to carry out the command requested by the user. For example, to change the 
channel on the main tuner 30, the controller 20 would send a suitable main channel control 
signal 20a to the main tuner 30. Details of implementing a tuner than can be tuned by 

25 applying a channel control signal are well known to persons skilled in the relevant arts. 

Similarly, to change the volume on the monitor 31, the controller 20 would send a suitable 
volume control signal (not shown) to the monitor 31. 

In addition to the conventional command keys 27a-d, 28a-b, the remote 
control 25 also includes a remember button 26, which is used to actuate the remember mode. 

30 When the remember button 26 on the remote control 25 is pressed, a unique command code 
25a corresponding to the remember button 26 is sent from the remote control 25 to the 
controller 20. Operation of the controller 20 upon receipt of this remember command code is 
described below. 
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Another example of a suitable function for implementing fingerprinting, also 
developed by Dimitrova, is image isolation. Image bitmaps may be used to generate a shape- 
dependent feature vector, which is applied to a back propagation neural network (BPNN). 
The feature vector provides greater emphasis on overall image shape while being relatively 

5 insensitive to variability problems. In the technique for isolating image regions, connected 
component structures are defined based on the edges detected. Since edge detection produces 
far fewer pixels overall than binarizing the entire field occupied by a symbol, the process of 
generating connected components can be much more rapid. The selection of feature space 
also enhances recognition speed. With simulated BPNNs, the size of the input vector can 

10 seriously affect throughput. Care should be taken to be selective with regard to the 

components used from the selected feature space. Of course, heterogeneous feature spaces 
may be formed by combining mixes of different metrics such as moments and line-segment 
features. Also, computational economies may be realized where the selected features share 
computational steps. 

!5 A chosen feature vector is generated by the appropriate image analysis. A 

variety of these can be used. A number of different feature spaces are suitable for use with 
this approach. Examples of some suitable feature spaces, which are size and rotation 
invariant and are considered particularly suitable to video image classification using a BPNN 
classifier, are described in detail below. 
20 As shown in Figs 3A - 3D a first feature space is derived from an original 

image 150 by, for example, forming a binary version 155 of the image 150, locating feature 
points 165-169 of the binarized image 155, and extracting those feature points 165-169 Note 
that while only a small number of feature points are illustrated, a larger number would be 
used in practice. Referring now to FIGS. 4A and 4B, a Delaunay triangulation (FIG. 4A) or a 
25 Voronoy diagram (FIG. 4B) is derived from the feature points 12. An image processor (not 
shown) performs the triangulation and then, for each triangle 1-6, generates an inventory of 
the internal angles. It then uses this inventory to generate a histogram of the angles, as shown 
illustrated in Fig. 5A. The histogram simply represents the frequency of angles A, B, and C 
of a given size range in the set of triangles 1-6 defined by the triangulation. Note that other 
30 triangulation methods or polygon-generating methods can be used. For example, referring to 
FIG. 4B, a set of Vorony polygons 17 and 18 can be used to define a set of angles A', B\ and 
C, each associated with a vertex 14 of the Voronoy diagram. The angle histogram that results 
serves as the feature vector for the particular character from which the features points were 
derived. 
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fingerprint compare unit 42 detects that the fingerprint of the A/V signal 40a is sufficiently 
similar to the fingerprint stored in the fingerprint memory 41, the fingerprint compare unit 
will send a match-detected signal 42a to the controller 20. 

With most pattern matching techniques, an exact correspondence with the 
5 stored fingerprint is not required to generate a fingerprint match - the fingerprints need only 
be sufficiently similar. For example, in a baseball game, if a fingerprint is stored when a team 
with white uniforms is fielding and two blue-uniformed runners are on base, a fingerprint 
when the blue-uniformed team is fielding would be sufficiently similar to result in a match. 
The pattern matching technique preferably includes a confidence measure that indicates the 
10 probability of a match. When the confidence level exceeds a confidence threshold, a match is 
declared. 

Optionally, the fingerprint compare unit 42 can adjust the confidence 
thresholds as a function of time. For example, because most commercials occur in 30 second 
increments, the fingerprint compare unit 42 can decrease the required confidence measure for 

15 a match detection at 60, 90, and 120 second intervals after the first channel change, so as to 
increase the probability of detecting a match at those intervals. Alternatively, instead of 
relying on fixed times, the system can learn the expected commercial intervals for a given 
program. The techniques for detecting when the original program has returned to the original 
channel may be combined with conventional commercial-recognizing techniques by 

20 changing the confidence threshold when the commercial-recognizing process determines that 
a commercial is likely to have ended. 

Optionally, the fingerprint compare unit 42 can slowly decrease the confidence 
threshold required for a match as the time from the first channel change increases, so that as 
the commercial lasts longer and longer, it becomes easier and easier to return to the original 

25 channel. 

Optionally, the fingerprint compare unit 42 may check for matches against a 
family of fingerprints, and generate a match-detected signal 42a when any member of that 
family of fingerprints is matched. This may be useful for programs that have multiple sets 
(e.g., the sitcom "Frasier", in which most scenes take place in either the living room, the 
30 kitchen, or the studio). When this option is implemented and the remember button is pressed, 
a fingerprint of the scene is captured and stored in the fingerprint memory 41. A database of 
fingerprint families (not shown) is then checked. If the captured fingerprint appears in the 
database, then the fingerprint compare unit 42 compares the fingerprint of the A/V signal 40a 
to each fingerprint in the captured fingerprints family. A match-detected signal 42a is 
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Once the system has set the aux tuner 40 to the desired channel and stored a 
fingerprint of the desired program, the user is free to change channels as desired. This user 
accomplishes this by pressing the channel change buttons 28. Anytime the user changes a 
channel by pressing a channel change button 28, the remote control 25 will issue a 

5 corresponding channel change command, which is received by the controller 20. 

The controller 20 tests to determine whether a channel change command has 
been received. If a channel change command has been received, the controller 20 changes the 
channel on the main tuner by issuing a main channel control signal 20a to the main tuner 30. 
This causes the channel displayed on the monitor 31 to change in accordance with the 

10 channel change button 28 on the remote control 25 pressed by the user. The user then sees the 
channel changing on the monitor 31 in the same way that it would ordinarily change on an 

ordinary television set. 

Importantly, when the channel setting of the main tuner 40 is changed, the 
channel setting of the aux tuner 40 is not changed, and the aux tuner remains set to the 
1 5 original channel that was being viewed when the remember button was pressed. Because of 
this, the fingerprint compare unit 42 can monitor the original channel and wait for a 
fingerprint that matches the fingerprint stored in the fingerprint memory 41. 

If, a channel change command is not detected, processing continues at a next 
step where the confidence thresholds are adjusted as required, (e.g., as a function of time, as 
20 described above). Next, a test is performed to determine whether a fingerprint match has been 
detected by the fingerprint compare unit 42. For example, if the remember button was 
pressed while a baseball game was being displayed, the fingerprint stored in the fingerprint 
memory 41 would characterize video or audio components of the baseball game. During 
commercials, the fingerprint of the A/V signal 40a generated by the aux tuner 40 will not 
25 match the fingerprint stored in the fingerprint memory 41, and processing returns to continue 
testing for channel change commands and fingerprint matches. When the baseball game 
returns, however, the fingerprint of the A/V signal 40a will match the stored fingerprint. 
When a match is detected, the fingerprint compare unit 42 will generate a match detected 
signal 42a, and send that signal to the controller 20. Preferably, detection of matches is 
30 inhibited until at least one channel change has occurred, so that a matches will not be acted 
upon before channel surfing has begun. 

When the controller 20 receives the match-detected signal 42a from the 
fingerprint compare unit 42, processing continues at a next step where the main tuner channel 
is set to match the channel setting of the auxiliary tuner 40. Preferably, this is accomplished 
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additional fingerprints may be collected as the dwell time continues, and a channel return 
may be executed when any stored fingerprint is subsequently matched. Preferably, 
fingerprints that immediately precede a channel change (e.g. by less than 5 seconds) should 
be discarded to avoid storing a fingerprint of a commercial. 
5 The present invention may be applied to media other than television, such as 

radio and Internet broadcasts. Of course, when an audio-only format such as radio is used, 
only the audio information would be available to implement the fingerprinting storage and 
comparison. The present invention may also be used in picture-in picture systems to switch 
picture windows when a return to the original program is detected. 
10 While the present invention has been explained in the context of the preferred 

embodiments described above, it is to be understood that various changes may be made to 
those embodiments, and various equivalents may be substituted, without departing from the 
scope of the invention, as will be apparent to persons skilled in the relevant art. 
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6. The method of claim4, wherein the confidence threshold is adjusted in said 

detecting step (42) so that a probability of detecting a match will increase gradually with time 
subsequent to a first channel change. 

5 7. The method of claim 1 , wherein in said storing step (41), the first portion of the 

first television program is a portion that is broadcast at substantially a same time as the 
storage initiation condition. 

8. The method of claiml, wherein in said storing step (41), the first portion of the 
10 first television program is a portion that is broadcast at least five seconds prior to a time when 

the storage initiation condition. 

9. The method of claim 1 , wherein the data that characterizes the first portion in 
said storing step (41) and the data that characterizes the second portion in said detecting step 

15 (42) are computed based on image angles in video data. 

10. The method of claim9, wherein the computation based on image angles 
includes Delaunay triangulation. 

20 11. The method of claim 1 , wherein the data that characterizes the first portion in 

said storing step and the data that characterizes the second portion in said detecting step are 
computed based on audio data. 

12. The method of claiml, wherein the data that characterizes the first portion in 
25 said storing step (41) and the data that characterizes the second portion in said detecting step 

(42) are computed based on motion detected from video data. 

13. a method as claimed in claiml, wherein the recognizing step (20) includes the 
step of: 

30 accepting (25) a storage initiation command (26) from a user while the user 

receives information via the first channel, with the storage initiation condition including the 
storage initiation command being accepted. 
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